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GENERALIZED  CORRELATIONS  IN  THE  SINGULAR  CASE 


Ashis  Sen  Gupta 

1.  Introduction.  When  the  covariance  matrix  is 
singular,  the  usual  expressions  for  the  multiple,  partial, 
canonical  [see,  e.g.,  Rao  (1973)]  and  some  generalized 
canonical  correlations  [for  a  review,  see  Sen  Gupta  (1979)] 
need  to  be  revised.  Tucker,  et  al.  (1972),  Khatri  (1976) 
and  Rao  (1979),  (1980)  have  provided  formulae  for  some  of 
these  correlation  coefficients  in  the  general  case  by  using 
g-inverses.  A  review  of  their  results  on  multiple,  partial 
and  canonical  correlations  is  given  first.  Next,  it  is 
shown  that  there  exists  a  general  representation  which 
covers  several  generalized  canonical  correlations  and  as 
special  cases  the  multiple,  partial  and  canonical  correla¬ 
tions,  too.  Then  a  general  theory  is  formulated  which  deals 
with  the  singular  case  for  the  representation.  Previous 
results  on  multiple,  partial  and  canonical  correlations 
follow  as  special  cases  of  this  theory.  Further,  appro¬ 
priate  formulae  are  also  provided  through  this  formulation 
for  various  generalized  canonical  correlations  in  the  singu¬ 
lar  case.  Finally,  the  numbers  of  various  critical  genera¬ 
lized  correlations  are  derived  for  the  general  case. 

2 .  Multiple,  partial  and  canonical  correlations  in 
the  singular  case.  Let  R  =  (R^:...:Rp)  be  the  correlation 
matrix  of  p  variables.  Further,  let  R  =  (r1^)  =  (T^:...:T  ) 
be  any  g-inverse  of  R.  Define,  RR  =  Q=(Q^ : . . . :Q  ) .  Let 


I  have  the  unit  vector  e.,  as  its  i-th  column,  i  =  1,..., 
P  ^ 


Result  1.  The  squared  multiple  correlation  of  x^  on 

X_  , .  .  .  ,X  is 
2  P 


R 


2 

1* (2. . .p) 


1  if  Q1  ^ 

11  “1 

1  -  (r  )  if  Qx  =  e1  . 


Result  2.  The  partial  correlation  between  X^  and  X2 

eliminating  X-  ,  X,  ,  •  •  .  ,X  is 

o  4  p 


r12‘  (34 . . .p)  =  0  if  Qx  *  ex  and  Q2  =  e2  or  if  Q1  =  and 

=  1  if  Qx  e1  and  Q2  /  e2 
12  11  22  1/2 

=  -rx  /(r  r  )  if  Qx  =  e±  and  Q2  =  e2  ' 

Let  and  X2  be  two  sets  of  variables  with  the  joint 
dispersion  matrix  L ,  partitioned  accordingly. 

Result  3.  The  squared  canonical  correlations  are  the 
non-zero  roots  of  the  determinantal  equation 

^11^12^22^21  “  P  *1  =  0  where  and  1^2  are  any 
g-inverses  of  Z22  and  Z.^  respectively. 

For  proofs  and  further  discussions  on  the  results 
see  Rao  [(1979) ,  (1980).] 

3.  Generalized  canonical  correlations  in  the 
singular  case.  Canonical  correlations  have  been  gener¬ 
alized  in  various  ways.  Formulae  in  the  general  case 
will  be  provided  here  for  those  obtained  by  extending 
the  concepts  of  tests  of  independence  for  two  sets  of 


3 


variates — giving  rise  to  partial,  part  and  bipartial  canon¬ 
ical  correlations  [see  Timm  (1975)  pp.  352-353)  and  g^^  -  ,  g2 
bipartial  canonical  correlations  [see  Lee  (1978)]  and 
some  association  measures  [see  McKeon  (1965)  ,  pp.  16-19] . 
Various  other  authors  [Horst  (1961) ;  Edgerton  and  Kolbe 
(1936);  Wilks  (1938);  Lord  (1958)]  arrived  at  the  same 
solution  as  that  of  McKeon  for  the  particular  case  of  a 
single  variable  per  set.  Appropriate  formula  will  also  be 
provided  for  the  new  generalized  canonical  correlation 
arising  out  of  the  concept  of  minimum  generalized 
variance  [proposed  by  Anderson  (1958)  Problem  5, 
pp.  305-306  and  derived  by  the  author  (see  SenGupta  (1979)} 
under  constraint  of  equi-correlation  structure  of  the 
generalized  canonical  variables] . 

Let  X  =  (X.  , . . . ,X,  )  ,  X.  :p.xl,  p  +...+  p  =  p, 

A.  K  1  1  I  K 

Disp(X)  =  I,  Cov  (X.  ,  X.)  =  E.  .  and  non-zero  ps  be 
k  1  3  13 

the  generalized  canonical  correlations.  Starting  with  the 
defining  equations  it  can  be  easily  seen  that  for  all  the 
above  cases,  the  generalized  canonical  correlations 

are  obtained  from  the  eigen  values  of  kE*  in  the  metric 
of  i*e*  from  the  solutions  of 

(3.1)  lkZ*  -  p*  kZ*|  =  0 

where  p*  =  1  +  (k  -  l)p  and  kE*  is  a  diagonal  super  matrix 
with  elements  Et^  i  =  l,...,k.  In  the  notation  of  Lee, 
kE*,  with  k  =  2,  is  the  covariance  matrix  of  the  residual 
vectors  (e^  34,  e2  35)  and  (e]L  34,  e2  35)  for  the  gx~  and 
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g2-bipartial  canonical  correlations,  respectively.  In  the 

notation  of  Timm,  kE*  =  I.3,  s1(2*3)  and  Z  ( 1 • 4 )  (2*3)  with 
k  =  2  for  partial,  part  and  bipartial  canonical  correlations 
respectively.  Also  for  McKeon's  and  the  new  generalized 
canonical  correlations,  =  ^E. 

Theorem .  The  generalized  canonical  correlations,  for  the 
methods  quoted  above,  are  given  by  p  =  (p*  -  l)/(k  -  1) 
where  ps  are  the  non-zero  roots  of  |kE*kE*-  “  P*l|  =  0,  kE*j 
being  any  g-inverse  of  kE*. 

Proof.  First  note  the  representation  (3.1).  Consider 
next  the  following  Lemmas. 

Lemma  1 .  Let  A  be  a  hermitian  matrix  of  order  n  and 
rank  s,  and  B  be  non-negative  definite  matrix  of  order  n 
and  rank  r  such  that  S( A)  c  S(B)  f where  S  (M)  represents 
the  vector  space  spanned  by  the  column  vectors  of  M] . 


Then 


(i)  There  exists  a  matrix  L  of  order  nxr  such 


that  L"AL  =  A,  L^BL  =  Ir,  where  A  is  a  diagonal  matrix 
with  s  non-zero  elements,  some  of  which  may  be  repeated 
and  Ir  is  the  identity  matrix  of  order  r. 

(ii)  The  non-zero  elements  of  A  are  the  same  as 
the  roots  of  the  determinantal  equation,  |AB  -  Al|  =0 
with  repetitions  allowed,  for  any  g-inverse  B  of  B. 
Proof  of  Lemma  1 .  See  Lemma  3,  Rao  (1979). 

Lemma  2.  S(kE*)  c  S(kE*)  . 

Proof  of  Lemma  2.  Note  that  S(E*^)  c  S(E*i)  for  all  the 
kE*  considered  above.  This  follows  immediately  from  the 


-»■*'  « .7  » 


5 


result  [see  Proposition  3.31,  pp.  3.15-3.16  of  Eaton 


that,  if  A  = 


11 

A12 

21 

A22 

£  0  then  N(A22)  c  N(A12)  and 


and  N(A^)  c  N(A2^)  where  N(M)  is  the  null  space  of  M. 

Then,  there  exist  matrices  B ..  such  thatB..E*;-  =  E*  .  ,  i,j  =  1,. 

13  ID  3  3  13 

Hence,  there  exist  matrices  such  that  <C^ . .  .C^)  ^Ejj  =  C  ^E 
which  proves  Lemma  2. 

Coupling  Lemma  2  with  Lemma  1  proves  the  Theorem. 

Note:  For  k=2,  if  p^  =  1,  p2>l  and  if  p^  >  1,  p2  >  1 

then  we  have  the  cases  of  multiple  and  canonical  correla¬ 
tions  respectively.  Further,  with  k  =  2,  consideration 
of  residual  variables  leads  to  partial  correlation.  Thus 
the  above  Theorem  unifies  the  Results  1  through  3  and  also 
considers  simple  (and  not  squared)  multiple,  partial  and 
canonical  correlations. 

4.  Numbers  of  critical  generalized  correlations. 


Let  A  and  B  be  two  hermitian  matrices  and  B  be  non-negative 
definite.  If  A  is  a  constant  and  v  a  vector  such  that 


Av=ABv,  Bv  ^  0,  then  A  is  called  a  proper  eigen  value  and 
v  a  proper  eigen  vector  of  A  with  respect  to  B.  In  the 
context  of  Lemma  1,  the  elements  of  A  are  called  the  proper 
eigen  values  and  the  corresponding  columns  of  L,  the  proper 
eigen  vectors  of  A  with  respect  to  B.  For  the  generalized 
correlations,  we  consider  from  (3.1)  only  the  proper  eigen 


&  * 


values  of  ^E*  with  respect  to  ^E£  .  Also  note  that  for 
k  2,  1  and  -1/  (k  -  1)  are  the  maximum  and  minimum 
possible  values,  respectively,  for  the  generalized  cor¬ 
relations.  Let  be  the  super  off-diagonal  matrix 

such  that  ,  E*=  .  E*  +  ,  E*  .  Also  let  R(M)  denote  the 
k  k  d  k  od 

rank  of  the  matrix  M. 

Lemma  3 .  The  numbers  of  zero,  unit  and  -l/(k  -  1) -valued 
generalized  correlations  are  given  by  r  -  R(kE*d) , 
r  -  Rf^E*^  -  (k  -  1)^E^]  and  r  -  R(^E*)  respectively, 
where  r  =  R(kE*). 

Proof ;  The  proof  follows  by  rewriting  (3.1)  as 

lkE*l  “  XkEdl  =  0  where 

(kEl  '  A)  =  [kEod  '  (k  -  1)p3'  [kEod  -  (k  ~  x)kEd'  <k  -  D(P  "  1 
and  [^E*,  (k  -  1 )  P  +  1]  for  the  zero,  unit  and -l/(k  -  1) -valued 

generalized  correlations  respectively  and  noting  the  one-one 

relationship  between  A  and  p  . 

Acknowledgements .  I  am  grateful  to  Professor  C.  R.  Rao 

and  Professor  T.  W.  Anderson  for  their  kind  remarks  on  the 

subject. 
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